{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Mock Classifier With Custom Endpoints Model\n",
    "\n",
    " * Wrap a basic python model for use as a prediction microservice in seldon-core\n",
    "   * Run locally on Docker to test\n",
    "   * Deploy on seldon-core running on minikube\n",
    "   * Example of using custom endpoints that are scraped by prometheus\n",
    " \n",
    "## Depenencies\n",
    "\n",
    " * [Helm](https://github.com/kubernetes/helm)\n",
    " * [Minikube](https://github.com/kubernetes/minikube)\n",
    " * [S2I](https://github.com/openshift/source-to-image)\n",
    "\n",
    "```bash\n",
    "pip install grpcio-tools\n",
    "```\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wrap model using s2i"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test locally using REST"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!make build_rest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker run --name \"mock_classifier_with_custom_endpoints_rest\" -d --rm \\\n",
    "    -e PREDICTIVE_UNIT_SERVICE_PORT=5000 \\\n",
    "    -p 5000:5000 -p 5055:5055 seldonio/mock_classifier_with_custom_endpoints_rest:1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!cd ../../../wrappers/testing && make build_protos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Call the prediction endpoint.\n",
    "# Send some random features that conform to the contract\n",
    "#\n",
    "!python ../../../wrappers/testing/tester.py contract.json 0.0.0.0 5000 -p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Call the custom endpoint.\n",
    "# In this example its used for the prediction call count.\n",
    "#\n",
    "!curl \"http://localhost:5055/prometheus_metrics\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker rm -v \"mock_classifier_with_custom_endpoints_rest\" --force"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test locally using GRPC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!make build_grpc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker run --name \"mock_classifier_with_custom_endpoints_rest\" -d --rm \\\n",
    "    -e PREDICTIVE_UNIT_SERVICE_PORT=5000 \\\n",
    "    -p 5000:5000 -p 5055:5055 seldonio/mock_classifier_with_custom_endpoints_grpc:1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Call the prediction endpoint.\n",
    "# Send some random features that conform to the contract,\n",
    "# using NDArray.\n",
    "#\n",
    "!python ../../../wrappers/testing/tester.py contract.json 0.0.0.0 5000 -p --grpc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Call the prediction endpoint.\n",
    "# Send some random features that conform to the contract,\n",
    "# using Tensor.\n",
    "#\n",
    "!python ../../../wrappers/testing/tester.py contract.json 0.0.0.0 5000 -p --grpc --tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Call the custom endpoint.\n",
    "# In this example its used for the prediction call count.\n",
    "#\n",
    "!curl \"http://localhost:5055/prometheus_metrics\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!docker rm -v \"mock_classifier_with_custom_endpoints_rest\" --force"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test in Minikube"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Start minikube\n",
    "#\n",
    "# NOTE this example was tested using minikube v0.25.2 version.\n",
    "# There was an issue with later minikube versions that caused\n",
    "# the s2i build to hang.\n",
    "#\n",
    "!minikube start --memory=8096 --feature-gates=CustomResourceValidation=true --extra-config=apiserver.Authorization.Mode=RBAC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Create a cluster-admin role binding\n",
    "#\n",
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!helm init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Start seldon-core and analytics\n",
    "#\n",
    "!helm install ../../../helm-charts/seldon-core-crd --name seldon-core-crd  --set usage_metrics.enabled=true\n",
    "!helm install ../../../helm-charts/seldon-core --name seldon-core\n",
    "!helm install ../../../helm-charts/seldon-core-analytics --name seldon-core-analytics \\\n",
    "    --set grafana_prom_admin_password=password \\\n",
    "    --set persistence.enabled=false \\\n",
    "    --set prometheus.service_type=NodePort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!eval $(minikube docker-env) && make build_rest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Create moldel deployment\n",
    "#\n",
    "!kubectl create -f model-deployment.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Wait until ready (replicas == replicasAvailable)\n",
    "#\n",
    "!kubectl get seldondeployments seldon-deployment-example -o jsonpath='{.status}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!cd ../../../util/api_tester && make build_protos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Make prediction request.\n",
    "#\n",
    "!python ../../../util/api_tester/api-tester.py contract.json \\\n",
    "    `minikube ip` `kubectl get svc -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].spec.ports[0].nodePort}'` \\\n",
    "    --oauth-key oauth-key --oauth-secret oauth-secret -p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Query prometheus for the custom metrics\n",
    "#\n",
    "# A \"custom_service\" is run on port \"5055\" that\n",
    "# is defined in model code \"MeanClassifier.py\".\n",
    "#\n",
    "# Prometeus set to scrape the model pod on port \"5055\"\n",
    "# by the annotations in the \"model-deployment.json\" manifest.\n",
    "#\n",
    "!curl -s \"$(minikube ip):$(kubectl get svc prometheus-seldon -o jsonpath='{.spec.ports[0].nodePort}')/api/v1/query?query=predict_call_count\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#\n",
    "# The prometheus metrics can also been seen in a browser\n",
    "# using the url shown after running this cell.\n",
    "#\n",
    "# In the prometheus ui execute the expression \"predict_call_count\"\n",
    "#\n",
    "!echo \"http://$(minikube ip):$(kubectl get svc prometheus-seldon -o jsonpath='{.spec.ports[0].nodePort}')\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Clean up\n",
    "#\n",
    "!minikube delete"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
